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1.  Introduction 

Suppose  X1,...(Xp  are  Independent  observations  from  a  density  f  with 

support  contained  In  [0,*].  We  consider  estimating  f(x)  by  a  quantity 
having  the  rational  form 

f(x)  -  (PQ  ♦  2  ^  Pj  cosJx)/|l-«e1x|2,  (1.1) 

A 

where  |«J<1  and  the  Pj's  are  explicit  functions  of  Xj,...,Xn.  An 
estimator  as  In  (1.1)  bears  an  obvious  resemblance  to  autoregressive  moving 
average  (or  ARMA)  spectra,  and  will  thus  be  referred  to  as  an  ARMA 
estimator. 

Apparently,  ARMA  type  probability  density  estimators  have  not 
previously  been  considered  In  the  statistical  literature.  Parzen  (1979) 
and  Carmichael  (1984)  have,  however,  proposed  autoregressive  (or  AR)  type 
estimators  of  the  form 

f(x;p)  •  cjl-«1e1x-...-£peipx|"Z. 

A 

Carmichael  (1984)  obtains  a  consistency  result  for  f(»;p)  by  allowing  the 
AR  order,  p,  to  tend  to  Infinity  at  a  certain  rate  with  the  sample  size, 
n.  In  the  current  paper,  the  AR  order  Is  fixed  at  1  and  the  MA  order  tends 
to  Infinity  with  n.  The  motivation  for  the  latter  scheme  Is  based  on  the 
numerical  analytic  device  known  as  the  e^-transform,  \rfiich  In  turn  Is 
related  to  the  notion  of  a  generalized  jackknife. 
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Hart  and  Gray  (1985)  studied  the  use  of  ARMA  representations  In 
approximating  (rather  than  estimating)  density  functions.  Their  results 
characterize  the  Integrated  squared  bias  of  (1.1)  In  a  number  of  different 
situations.  The  current  paper  greatly  generalizes  the  results  of  Hart  and 
Gray  and  also  considers  the  mean  Integrated  squared  error  (MISE)  of  (1.1). 
It  Is  shown  that  there  exist  quite  general  conditions  under  which  an 


estimator  of  the  form  (1.1)  has  asymptotically  smaller  MISE  than  does  any 
window-type  Fourier  series  estimator.  For  example,  this  optimality 
property  obtains  when  f'(0+)*0,  f'(*-)-0,  and  f"  Is  square  Integrable. 

More  generally,  we  obtain  results  on  the  behavior  of  (l.l)'s  MISE  when  the 
Fourier  coefficients  of  f  are  either  approximately  exponential  or  regularly 
varying  at  Infinity. 

The  paper  will  be  ordered  In  the  following  way.  In  Section  2,  the 
estimator  to  be  studied  Is  defined,  and  a  number  of  motivations  for  Its 
use  are  given.  Section  3  contains  some  basic  results  concerning  the  MISE 
of  the  ARMA  estimator.  The  asymptotic  MISE  of  the  estimator  Is  studied  In 
Sections  4  and  5  under  the  aforementioned  conditions  on  the  Fourier 
coefficients  of  f.  In  Section  6  It  Is  shown  that  trfien  f  is  smooth  (In  a 
well-defined  sense)  at  one  endpoint  of  Its  support  but  not  at  the  other, 
then  the  ARMA  estimator  Is  asymptotically  superior  to  traditional 
window-type  estimators.  Cross-validated  smoothing  of  ARMA  estimates  Is 
addressed  In  Section  7,  and  some  summary  remarks  are  given  In  Section  8. 


2.  The  Proposed  Estimator 


Let  Xj,....Xn  be  •  randOM  sample  from  a  density  f  with  support 
contained  In  [0,«].  Me  shall  assume  that  f  has  the  Fourier  series 

f(x)  -  (l^jcosj*).  0  <  x  <  *,  (2.1) 

where  ^  ■  Jj  cosjx  f(x)dx.  Cencov  (1962),  Kronmal  and  Tarter  (1968), 

Hall  (1983)  and  others  have  Investigated  density  estimators  of  the  form 

f„(x;m)  -  «-1  (1+2^  ♦jCOSjx),  0  <  x  <  *, 

where 

-1  n 

•j  ■ "  J,  co,J*k- 

The  estimators  to  be  studied  here  are 

f„(x;m,«)  -  f„(x;m)  ♦  (2/«)  Real  ,  (2.2) 

L  1  *  «exp(1x)  _ 

where  -1  <  «  <  1.  The  pair  (m,a)  Is  the  smoothing  parameter  of  the 
estimator  and  can  be  chosen  from  the  data  by  cross-validation.  Me  will 
return  to  this  point  In  Section  7. 

Before  further  discussion  of  the  proposed  estimator,  we  should  justify 
using  the  cosine  basis  as  opposed  to  a  basis  with  both  cosine  and  sine 
functions.  This  study  was  In  part  motivated  by  the  problem  of  estimating 
animal  abundance  using  the  line  transect  method  (see  Gates  and  Smith  1980 
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and  Crain,  Burnham,  Anderson,  and  Laake  1979).  The  density  function 
estimated  in  this  setting  is  typically  assumed  to  be  monotone  decreasing  on 
(0,n).  Since  f(0)*f(«),  the  periodic  extension  of  f  is  discontinuous  at  0 
and  n,  and  hence  a  cosine-sine  Fourier  series  estimator  will  perform  poorly 
near  these  two  points.  As  shown  by  Hall  (1983),  though,  the  cosine  series 
estimator  is  not  adversely  affected  by  the  condition  f(0)  *  f(ic).  In 
kernel  estimation,  the  analogous  means  of  correcting  boundary  problems  is 
the  symmetrization  device  studied  by  Schuster  (1985). 

A 

There  are  a  number  of  ways  of  characterizing  the  estimators  fn(«;m,o). 


First,  it  is  clear  that  they  may  be  written 


(2.3) 


where  fL  *  p_^  and  the  p^'s  depend  only  on  a  and  the  ♦j’s.  Aside  from  the 
issue  of  positivity,  (2.3)  has  the  form  of  an  ARMA(l,m+l)  spectrum,  hence 
the  name  ARMA  probability  density  estimator.  The  form  (2.3)  suggests  that 

A 

fn(*;m,o)  is  well  suited  for  densities  with  large  "power"  at  either  0  or  * 
(but  not  both).  While  this  Is  so,  it  will  be  seen  In  Section  6  that  such 
an  observation  somewhat  understates  the  value  of  ARMA  estimators. 

A 

Perhaps  a  more  Interesting  way  of  characterizing  fn(*,m,o)  Is  In  terms 
of  the  generalized  jackknife  and  the  numerical  analytic  device  known  as  the 
ej -transform.  Using  (2.2),  It  is  easy  to  show  that,  for  m  >  1, 


.1  fFj*)  -  ae  F—  i ( *) 

f  (x;m,a)  «  %  1  1  ♦  2  Real! - 

V  i  -  «’* 


(?.<) 


V7VC 


>  V  9  »  f 


lA-  A**  a"  v  •'  ■  *  A*  A-**  ►  *  «  *  A 


where  Mx)  -  L  ♦<  e1^*,  and  Fft  s  0.  The  quantity 

*  j»l  J  u 

G-  (x)  »  (F  (x)  -  ae1x  F  1(x))/(l  -aeix)  Is  a  generalized  jackknife 

m9a  m  in- 1 

estimator  (as  defined  by  Schucany,  Gray,  and  Owen  1971)  of  the  function 

F(x)  -  I  ♦.e1^  . 
j»l  J 

The  problem  of  choosing  a  In  such  a  way  that  Gm  a(x)  has  smaller  bias  than 
Fm(x)  will  be  addressed  later. 

Defining  F^x)  *  EfF^x)),  we  have 

■  <FJ«>  -  -  «,x>- 

If  a  is  taken  to  be  ♦,„+!/♦„,»  then  (E(Gm  a(x)):m*l,2,...}  Is  the  ej- 
transform  of  the  sequence  (Fm(x)).  This  transform,  which  dates  at  least  to 
Altken  (1926),  Is  a  numerical  analytic  tool  used  for  accelerating  the 
convergence  of  a  sequence  to  Its  limit.  The  work  of  Shanks  (1955)  gives  a 
thorough  account  of  e^  and  the  more  general  ep-transform.  The  Interesting 
and  enlightening  paper  of  H.  L.  Gray  (1985)  demonstrates  the  close 
connection  between  many  numerical  analytic  methods  (including  the  ep- 
transform)  and  the  statistical  notion  of  jackknifing  to  reduce  bias.  For 
more  uses  of  the  en-transform  In  statistical  problems,  see  Gray,  Watkins, 
and  Adams  (1972)  and  Morton  and  Gray  (1984). 

A 

Also  of  Interest  Is  the  Fourier  series  of  fn(*;m,a).  From  (2.2), 

f  (x;m,a)  -  f  (x;m)  +  (2/x)  T  ♦  a^"1"  cosjx  . 

n  n  j»m+l  m 


V 


The  Fourier  coefficients,  ^(m.a),  of  the  ARMA  estimator  are  thus 

.  J  ■  0.1 . . 

A  A 

This  shows  that  fn(x;m,a)  will  tend  to  have  smaller  bias  than  fn(x;m)  when 
the  ^  decay  geometrically.  As  will  be  seen,  though,  geometrically 
decaying  ♦j's  are  but  a  subset  of  the  cases  In  which  fn(*;m,a)  affords  a 
bias  reduction.  Hart  and  Gray  (1985)  argue  that  ARMA  type  approximators 
are  very  generally  an  effective  means  of  counteracting  the  leakage  effect 

A 

Inherent  in  fn(*;m). 

3.  Mean  Integrated  Squared  Error  of  fn(«;m,o) 

Define  the  mean  Integrated  squared  error  of  the  estimator  f  by 
J(f,f)  ■  E  Jq  (f(x)-f(x))2dx.  The  MISE  will  be  used  as  a  basis  for 

*  A 

comparing  fn(«;m,a)  with  fn(»;m)  and  more  general  Fourier  series 
estimators. 

It  is  straightforward  to  show  that 

J(fn(*;m,a),f)  ■  (2/n)  [n-1  J  varfcosjXj)  +  a2(l-<x2)_1  var(cosmX1 )/n] 

*  (2/’VL  <wJ-v  •  <3'D 

where  var(cosjXj)  ■  (l+^gj)/2  -  ♦j  .  The  first  of  the  two  terms  in  (3.1) 

A 

Is  the  Integrated  variance  of  fn(*;m,a)  while  the  latter  is  the  integrated 
squared  bias.  Note  that  taking  a-0  in  (3.1)  yields  J(fn(*;m),f).  This  is 
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Important  since  It  makes  clear  that  one  may  always  choose  a  so  that  the 

WISE  of  an  ARMA  estimator  Is  no  bigger  than  that  of  fn(*;m). 

If  {affl  n:m,n  ■  1,2,...}  Is  a  sequence  satisfying  ja^  n|<l  and 

11m  Inf  (1-or  )>0,  then  as  n,m-H» 

n,m+-  m»n 

,,r(cosj*1)  $Xl(  Wftfr 

+  Ofn'1).  (3.2) 

In  this  event,  then,  the  effect  of  affl  n  on  the  variance  of  the  ARMA 

estimator  Is  asymptotically  negligible  as  n  and  m-*-.  It  will  be  seen, 

though,  that  there  are  also  cases  where  a  good  choice  of  o  _  Is  such  that 

m,n 

2  2  2  1 
m(l-cr  )+c  as  m,n-*«*.  In  such  cases  the  term  a£  „(l-cr  varfcosmX. )/n 
m,n  m,n  m,n  l 

Is  of  order  m/n  and  cannot  be  Ignored  In  asymptotic  considerations. 

Before  proceeding,  we  give  the  following  useful  lemma. 

Lemma  1.  Suppose  f  has  square  summable  Fourier  coefficients.  Then,  as 

m-*-, 

m 

T  var(cosjX.)  «  m/2  +  0(^m). 
j«l  1 

If  It  Is  further  required  that  the  ♦j's  be  absolutely  summable  the  term 
0(^m)  may  be  replaced  by  0(1). 


Proof:  Recall  that 


m 


m 


m 


X  var(cosjX.)  ■  m/2  +  (1/2)  L  -  .L  ♦i- 
j*l  1  j»l  £J  j*l  J 


If  the  ♦j's  are  absolutely  summable.  It  Is  clear  that  the  last  expressl 


on 
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is  m/2  +0(1).  By  the  Cauchy -Schwarz  inequality, 

iA  vii  -<  ^  <jl  ♦i//2  • 

from  which  the  rest  of  the  lemma  follows. 

4.  Densities  With  Nearly  Exponential  Fourier  Coefficients 

In  Section  2  it  was  mentioned  that  the  ARMA  estimator  should  perform 
well  for  densities  with  geometrically  decaying  4j's,  i.e.,  ones  for  which 
♦j  -  cexp(-aj)  as  j-»«.  In  this  section,  we  shall  investigate  the  more 
general  case  in  which  f  has  Fourier  coefficients 

♦j  -  e‘ajRp(j),  j  -  1,2,...,  (4.1) 

where  a>0  and  Rp  Is  a  regularly  varying  function  (defined  on  [1,»))  with 
index  p,|p|<«. 

A  A 

An  ideal  asymptotic  comparison  of  J (f n( •;m,a) ,f)  and  J(fn(*;m),f) 

*  ★  * 

would  be  to  investigate  Rp  *  Jn/Jn»  where  Jp  =^m1n^ 

and  =  m^n  J(fp( • ;m) ,f ) .  Instead  of  this  approach,  we  shall  choose  a  to 

A 

depend  on  at  most  m,  and  then  study  m^n  0(fn( *;m,am) ,f)  as  n+®.  The  am  to 
be  used  is  one  suggested  by  generalized  jackknife  theory.  According  to 
Schucany,  Gray,  and  Owen  (1972), 

E  -  rlW«»/<1-r»  -J,  V1jx 

if  r  *  £  *,e^x/  I  4>.e^x  .  When  (4.1)  holds,  the  last  quantity  is 

j*m+l  ^  j=m  J 

asymptotic  to  (♦  ./♦  )e1x  as  m-*«.  Referring  to  (2.4),  then,  a 
m+1  m 


.  V. 
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reasonable  choice  for  am  would  seem  to  be  Recall  that  with 

m  m+i  m 

«  *  ♦m+l^m  »  E^Gm,o^x^  the  “transform  of  Fm(x)  (see  Section  2). 

A  A 

,  Defining  B(f,f)  to  be  the  integrated  squared  bias  of  f,  it  follows 
from  Theorem  4  of  Hart  and  Gray  (1985)  that  whenever  am  =  i’m+i/i,ni+e~a(a>0) 

lim  B(f  (*;m,a  ),f)/B(f  (*;m),f)  =  0. 

OT+co  n  m  n 

When  the  satisfy  (4.1)  we  have  the  following  more  precise  result. 

As  a  matter  of  convenience,  is  taken  to  be  e"a  in  the  remainder 
of  this  section. 

Lemma  2.  Let  f  be  a  density  with  Fourier  coefficients  as  in  (4.1).  Then, 
as  m-*«>, 

B(fn(*;m),f)  -  (2/«)e‘2a(fn+1)R2(m)(l.e~2a)-1, 
B(fn(*;m,e_a),f)/B(fn(.;m),f)  ~  Am  = 

(l-e~2a)  Jq  (Rp(j+m+l)/Rp(m)-l)2e‘2aj,  and 

lim  A  =  0. 
m-*»  m 

Proof:  Define 

=  L  e"2aj  R2(j+m+l)/R2(m). 

J*0  P  P 

Since  R  is  regularly  varying,  it  follows  that  R2(m+l)/R2(m)-*-l  as 


11 


2a 

Hence,  for  0<e<e  -1,  all  m  sufficiently  large  and  each  j>0 

e-2ajR2(j+m+l)/R2(m)  <  e“2aj‘(l+e)j+1  =  (l+e)exp[-j(2a-log(l+e))].  Since 

the  last  expression  is  summable,  dominated  convergence  immediately  gives 
-2a  -1 

S  >(l-e  )  ,  proving  the  first  part  of  the  lemma.  The  rest  of  the  lemma 

is  easily  obtained  by  taking  o  =  e  in  B(fn(*;m,o),  f)  (see  (3.1))  and 
arguing  as  we  just  did. 

Under  the  conditions  of  Lemma  2,  we  see  that  the  integrated  squared 
bias  of  fn(*;m,e  )  is  asymptotically  negligible  compared  to  that  of 

A  A  9 

f n( * ;m) .  Furthermore,  appealing  to  (3.2),  it  follows  that  J(f  (*;m,e“a),f) 

A 

is  asymptotically  no  larger  than  J(f  (*;m),f)  as  n,m—».  The  following 
theorem  gives  a  more  precise  result  concerning  the  MISE's  of  the  two 
estimators. 

Theorem  1.  Let  Rp  be  any  regularly  varying  function  (defined  on  [1,®)) 
with  the  representation,  for  all  t  sufficiently  large, 

o  ,t  e(x) 

Rp(t)  =  tpexp(c  +  JB  x  dx), 

where  p*0,  |c|<®,  and  B>1.  If  the  Fourier  coefficients  of  f  are 
♦j  =  e"a^Rp(j)  .  J  =  1.2,... , 

then 

0(fn(*;m),f)  =  (1/*)  [m/n+2t2(e2a-l)”1]  +  0(l/n)  +  o(*2)  (4.2) 

and 

J(fn(*;m,e_a),f)  =  (1/*)  Cm/n+2p242  m"2Ca] 

+  0(l/n)  +  o(*2  m-2),  (4.3) 
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where  Ca  =  e"^a(l+e”^a)(l-e”^a)”'*  .  Defining  m„  to  be  the  minlmizer 
s  n 

A  It  p  o  o 

of  0(f_(,;m) ,f)  and  m  to  be  the  minimizer  of  m/n  +  2p  rm  C.,  we  also 

n  n  In  3 

have 

*  -1 

0(fn(*;mn),f)  *  (2a*n)“  [logn+2ploglogn]  +  o(loglogn/n) , 
and 

J(fn(*;m*,e‘a),f)  =  (2ann)-1  [logn+2( p-l)loglogn] 

+  o(loglogn/n) . 

Proof:  Equation  (4.2)  is  an  immediate  consequence  of  Lemmas  1  and  2.  To 
obtain  (4.3),  note  that  for  all  m  sufficiently  large 

Rp(j+m+l)/Rp(m)-l  = 

(l+(j+l)/m)pexp(/J[+‘i+1  e(x)x_1  dx)  -  1  = 

(l+(j+l)/m)p  -  1  +  (l+(j+l)/m)p  [expf/J^*1  e(x)x_1  dx)  -  1] 


=  (j+l)m_1  +  (1  +  (j+l)/m)p[exp(c^m)  /J^+1  e(x)x-1  dx], 

where  1  <  b.  <  1  +  (j+l)/m  and  c.  is  a  number  between  0  and 

✓  "  j "  j 

Cj+1  e(x)x_1  dx. 

We  have 

KrTJ+1  eMx'1  dx|5ml(J+1)supj  e(x)  J 


and  since  Rp  Is  regularly  varying,  e(x)-*0  as  x-**  (see  Seneta  1976,  p.  3). 
Applying  Lemma  2,  the  above  results,  and  dominated  convergence  gives 

B(fn(*;m,e"a),f)  -  (2/*)e‘2a*2  m‘V  ^  (j+l)2e"2aj. 

Equation  (4.3)  follows  from  this  and  Lemma  1.  For  the  rest  of  the  proof, 
note  that  mn  Is  such  that  A(n,mn)<0  and  A(n,mn+1)>0,  where 

A(n,t)  *  (1+Rp(2t)e“2at)  -  2(n+l)R2(t)e‘2at. 

Since  A(n;»)  Is  eventually  continuous.  It  follows  that  for  all  n 
sufficiently  large  A(n,tn)  *  0  for  some  tp  e[mn,mn+l].  How  A(n,tn)  *  0 
Implies 

tn[l-plogtn/(atn)  -  (c+  fgn  e(x)x-1  dx)/(atn)] 

=(2a)~1log(n+l)[l-log(l/2  +  Rp(2tn)exp(-2atn)/2)/log  (n+1)]. 

This  implies  that  tfl  *  (2a)"1  logn  +o(logn).  Substitution  of  this 
expression  for  tn  Into  the  previous  equation  gives 

tn  *  (2a)_1[logn  +2ploglogn  +  o(loglogn)], 
where  we  make  use  of  the  fact  that  logL(x)/1ogx-*0  for  any  slowly  varying 

A 

function  L  (see  Seneta,  p.  18).  The  expansion  for  0(fp( *;mn) ,f)  follows 
upon  observing  that  mp  =  tp+0(l)  and  using  (4.2).  The  proof  for  the  ARMA 
estimator  follows  In  an  analogous  manner. 

4 

Theorem  1  gives  conditions  under  which  an  ARMA  estimator  has,  for  all 
n  bigger  than  some  nQ,  smaller  MISE  than  does  the  best  estimator  fn(*,m). 
The  Improvement,  though,  is  only  in  terms  of  second  order  efficiency.  Thi 


occurs  for  two  reasons.  First,  for  the  ♦j's  considered,  the  two  optimum 
MlSEs  are  dominated  by  their  variance  terms,  and  so  the  smaller  bias  of  the 
ARMA  estimator  Is  not  reflected  In  the  leading  WISE  term.  Secondly,  Vj’s 
of  the  form  (4.1)  with  p*0  are  not  near  enough  to  being  geometric  for  the 
ARMA  estimator  to  be  fully  effective.  The  following  theorem  provides 
conditions  under  which  an  ARMA  estimator  yields  a  first  order  type  of 

A 

Improvement  over  fn(*;m). 

Theorem  2  Suppose  the  density  f  has  Fourier  coefficients 


»  e‘aj  l(j)  ,  j  »  1,  2,...,  where 

L  Is  a  slowly  varying  function  such  that,  for  some  6>0,  the  quantity 
bm(j)  ■  exp(6m)[L(j+m+l)/L(m)-l]  satisfies 


Hm  I  *\!(j)e~2aj  *  b  ,  *1th  0  <  b  < 

m+-  j-0  m 


It  then  follows  that 


iim  o(fn(-;«ve"a),f) 

nlm  — r -  *  11m  mn/mn  »  a/(a+6), 

. ,  *  .  ..  n-n. 


J(fn(*;mn),f) 


where  m„  and  m„  are  as  In  Theorem  1, 
n  n 


Proof:  From  (3.1), 


B(f  (•  ;m,e"a) ,f)  -  (2/e)  42  T  ,  e_2a( j_,n) (L( j )/L(m)-l )2 
n  m  j*m+l 

-  (2/n)  42  e"2a  e"26m  T  b2(j)e‘2aj 
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-  (2/«)  L2(«)b*-24 

♦  o(42e"26*)  . 

Using  Iim  1  It  now  follows  that 

J(f_(*;».«”*).f)  -  (1/*)  im/n  ♦  2e'2(**6)"  L2(m)be“2*]  . 

n 

Proceeding  as  In  the  proof  of  Theorem  1, 

J(f_(«;«*,e'*),f)  -  ( l/«)(m*/n)  •  logn(2«(6+a)n)'1  ♦  o(logn/n). 
n  n  n 

The  result  follows  after  similarly  analyzing  the  NISE  of  fn(*;m). 

In  Hart  'nd  Gray  (1985),  It  was  claimed  that  ARHA  estimators  are  often 

more  parsimonious  than  Fourier  series  estimators.  Evidence  of  this  Is  seen 

In  Theorem  2.  In  fact,  under  the  conditions  of  this  theorem,  the  limits  of 

J(f„(*;m  ,e‘a), f)/J(f  (*;m.,),f)  and  m* /m  are  one  and  the  sane,  with  both 
n  n  n  n  n  n 

being  less  than  1.  An  example  of  ' s  satisfying  the  conditions  of 
Theorem  2  Is 

♦j  «  l/cosh(aj)  ,  j  ■  0,1 . 

It  Is  easily  verified  that  the  value  of  6  for  these  aj’s  Is  2a,  and  so  the 

WISE  of  fn( • ;m* ,e'*)  Is  asymptotically  only  1/3  that  of  fn(*;m). 

Furthermore,  this  savings  Is  obtained  even  though  the  ARHA  estimator  uses, 

* 

In  the  limit,  only  1/3  as  many  Fourier  coefficients  as  does  fn(»;m). 

Also  of  interest  Is  comparing  ARHA  estimators  to  general  Fourier 
series  estimators  of  the  form 

f  (x)  -  (l/x)  ( 1+2  T  w  (j)e.cosjx)  .  (4.4) 

w  j-1  n  J 
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Watson  (1969)  has  shown  that  the  minim*  WISE  among  such  estimators  Is 


Jn,opt 


-  ♦ 


<2/«i  y  _i 


j*l  var 


varfcosjXj) 
(cosjXj )  ♦  naj 


(4.5) 


The  following  theorem  shows  that  under  the  right  conditions  an  ARMA 
estimator  Is  better  than  any  estimator  of  the  form  (4.4). 


Theorem  3  Suppose  that  In  addition  to  the  conditions  of  Theorem  2 
| L( J ) j  <  A  <  -  for  all  j. 

It  then  follows  that 

where  m*  and  6  are  as  In  Theorem  2. 
n 


Proof:  It  Is  sufficient  to  show  that  Jp  opt  -  logn/(2na«).  To  do  this, 
we  proceed  as  in  the  proof  of  a  theorem  In  Section  4  of  Watson  and 
Leadbetter  (1963).  If  the  ♦j's  are  absolutely  summable.  It  Is  easy  to 
show  that 


n,opt 


<«*>  X 


j-1  U2(n-l)ej 


♦  0(l/n) 


We  have 


T  .  T  e~2aJ 

J-1  l«-2'(n-l')#J  j-1  l+2(n-l )e"^aJ 


< 


2  P 

<UA  ’  j-1  ri*2'(n-l).Jfl.2(n-l)e-?W1  5 


-j »-  »■  V- •- 
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(l^A2){(n-l)”1 


n  .  • 

y  1  ...  ♦  y 

j-l  [U2(n -1)*J]  j-c^l 


e*2aj> 


where  cn  Is  the  greatest  Integer  <  log(n-l)/(2a)  . 

Now,  y  e’2a^  <  (l-e‘2a)"1(n-l)”1.  Also,  since  L2  Is  slowly  varying, 

j-c  +1 
J  n 

o 

we  have  jL  (j)-*«  as  j-*-  (see  Seneta,  p.  18).  Hence,  there  exists  j  such 
that  for  all  j  >  jQ 

[l«*(n-l)#Jrl  <  [l+2(n-l)j"^e“2a^C]"1  (C  >  0) 

and  t  e  Is  monotone  decreasing  on  (J0,«).  It  follows  that  for  all  n 
sufficiently  large 

cn  cn 

(n-l)‘l  l  ri+2(n-l)^rl  <  (n-1)’1  J  [H2(n-l)j_1e’2ajC]'1 
j-l  J 


♦0(n“2) 


_i  log(n-l )/2a  ,  t  ,  , 

<  (n-1)  1  L  [l*2(n-l)t  le  C]  *dt  ♦  0(n  £) 
Jo 


<  1551=41 1\  r'du ♦  o<*-*> 

-  2a ( n- 1 )  0  TogTn-TT 


By  dominated  convergence,  the  previous  Integral  tends  to  0  as  n 
proof  of  the  theorem  Is  completed  by  showing  that 


The 


,-2*j 


-1 


— I  - 1 - —  *  (4a)  '  as  n  —  . 

logn  j«l  l*2(n-l)e-‘®J 
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The  previous  theorem  shows  thet  ARMA  estimators  are  fundamentally 
different  than  Fourier  series  estimators  of  the  form  (4.4).  The  full 
extent  to  which  this  Is  true  will  become  apparent  In  the  next  two  sections. 

Before  proceeding,  we  also  note  that  a  result  analogous  to  Theorem  1 
can  undoubtedly  be  obtained  for  densities  satisfying  ♦  .  cexp(-ajY), 

0  <  y  <  1.  For  these  densities.  Hart  and  Gray  (1985)  show  that  If 

®m  -  then  V*;w*am)  •*)/*(  V  *s 

5.  Densities  With  Regularly  Varying  Fourier  Coefficients 

In  a  certain  sense,  the  results  In  Section  4  are  not  particularly 

surprising.  The  Fourier  coefficients  of  fn(»;m,e  )  are  reasonably  well 

matched  to  those  of  the  underlying  density,  and  so  one  might  expect  the 

* 

ARMA  estimator  to  have  much  smaller  bias  than  fn(*;m).  Of  more  Interest  is 
to  Investigate  the  robustness  of  ARMA  estimators  to  departures  from  the 
approximate  exponential  model  for  the  ♦j's* 

In  this  section  we  study  the  case 

♦j  -  R.p(j)  .  j  -  1,2 .  (5.1) 

where  R_p  Is  a  continuous,  regularly  varying  function  of  index  -p  and 
p>1/2.  Included  in  such  cases  are  densities  with  Fourier  coefficients 

♦j  -  (H(j/a)p)_1  ,  j-0,1 .  (5.2) 

where  1/2  <  p  <  2.  More  generally,  any  set  of  square  summable  a.'s  that 

0 

decay  algebraically  (see  Watson  A  leadbetter  1963  and  Davis  1977)  satisfy 
(5.1). 

We  here  present  a  lemma  concerning  the  Integrated  squared  bias  of  ARMA 
estimators  when  (5.1)  holds. 
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Lwm  3  Let  the  a^'s  be  as  In  (5.1),  and  suppose  that  md-a^tc  >  0  as 


«**•.  Then 


jXiurvi‘*>2  *  m*i  r0  <(i+y)'p-e_cy)z<iy  ♦ 


Proof:  Since  the  4j's  are  regularly  varying,  *  /q  (1+y)"2p<1y 

«  — .  Also,  ♦’  JXi«2(J-’/(-.2)  ■  \  ■  /;  .-2cydy, 

There  exist  (see  Seneta,  pp.  19-20)  functions  (  and  4,  defined  on  (0,«), 
with  the  properties  4(t)  -  $(t)  as  t-*-  and  ♦( t )  <  a^+1  <  Mt)  for  all 


t>0.  By  assumption. 


l-(l+e)c/m  <  a  <  1  -  (l-e)c/m 


for  0  <  e  <  1  and  all  m  sufficiently  large.  It  follows  that 


1 :  “"jXi  *j  ■ 


11m  /I(m9HI)"1^(t)(l-(l-t)c/m)t”mdt 
m-*-  *  m 


11m  Jj  (♦(mu)/4m)(l-(l-c)c/m) 


m(u-l). 


Making  use  of  dominated  convergence,  the  last  limit  Is 


-P  -(l-e)c(u-l), 


Obtaining  a  similar  lower  bound,  and  using  the  fact  that  e  may  be  taken 


arbitrarily  small,  we  have 


Hm  J  V*"m  /(ma  )  -  Jg(l+y)'Vcydy. 
j-rn+l  J  m  m  u 


••  •*.  -*  «/ 


I 


The  lemma  now  follows  upon  combining  the  previous  results. 

Some  remarks  are  In  order  here.  First,  we  see  that  the  choice  of  <*m 
In  Lemma  3  Is  such  that  am~  1.  This  Is  required  since  the  ♦j ' s  In  (5.1) 
decay  to  zero  more  slowly  than  do  those  In  the  previous  section.  It  Is 
straightforward  to  show  that  If  (5.1)  holds  and  |amJ  Is  bounded  away  from 

A 

1,  the  limiting  Integrated  squared  bias  of  fn(*;m,am)  Is  the  same  as  that 

A 

of  fn(*;m). 

Now,  since  the  ♦j's  are  regularly  varying  with  Index  -p  <  -1/2,  It 
follows  that 

,  I  ,  ♦?/{•♦*>  *  (2P-1)'1  •  /„(l*y)‘2pdy.  (5.3) 

j-m+1  J  m  u 

A  A 

Hence,  the  Integrated  squared  biases  of  fn(*;m)  and  fn(*;m,afl|)  (with  am  as 

In  Lemma  3)  are  both  of  order  m#‘  as  m-*-.  However,  by  choosing  such 

that  m(l-a  )+p, 
nr 

*  (2 P-1)  JJ  ( (l+y)'P-e’py)2dy  <  1. 

Note,  though,  that  with  m(l-am)*p,  the  contribution  of  to  the  Integrated 
variance  of  the  ARMA  estimator  Is  not  Insignificant.  The  smaller  bias  of 
the  ARMA  estimator  does  not,  then.  Immediately  Imply  that  its  MISE  is 

A 

smaller  than  that  of  f  (•;m).  This  point  Is  Investigated  In  the  next 
theorem. 

Theorem  4  Let  the  Fourier  coefficients  of  the  density  f  be  as  In  (5.1), 
and  suppose  that  m(l-am)-*c  >  0  as  Then,  defining 

'p.e  '  /;m*y)‘p-«'cy)^y. 
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J(V*;m*am)’f)  -  n-1C(m/n)(H(2c)-1)  ♦  2m*2  Ip^] 


+  o(m/n  +  m**),  and 


*  *_1[m/n  +  2m^(2p-l)_1] 


♦  0(Jm/r\)  +  o(m*2) 


Furthermore,  If  mn  and  mn  are  the  minlmlzers  of,  respectively, 

[(m/n)(l+(2c)‘1)  +  2(2p-l)I  r  T  ,♦?]  and  J(fn( *;m) ,f ) ,  then 

P*c  j*m+l  J  n 

J(f„( *;m*,am*) ,f )/J(fn{ •;»„) ,f )  ~ 
n 

(m*/mn)(1+(2c)"1)  -  C(2p-l)Ip>c]1/(2p)(l+(2c)_1)1‘1/(2p) 

as  nx». 


Proof:  The  two  WISE  expressions  follow  from  (3,1)  upon  applying  the 
condition  m(l-a _)+c.  Lemmas  1  and  3,  and  expression  (5.3).  Now,  using 
(5.3)  and  the  first  part  of  Theorem  4, 

J(f  (.;m,a  ),f)  -  x'1C(m/n)(l+(2c)_1)  +  (2p-l)  I 

n  m  P»cj=m+1  j 

* 

If  mn  is  the  minimlzer  of  this  last  expression,  then  the  continuity  of  R_p 

implies  that  R2p(tp)  *  A^n"1  for  some  tp  e[m*,m*+l],  where 

A  «  2(2p-l)I  (l+(2c)"1)~1.  By  5°,  p.  21  of  Seneta,  there  exists  a 

pf  c 

regularly  varying  function  r  of  index  l/(2p)  such  that 


r(l/R_p(t))  -  t  as  t+«. 


-  «/  ’-'•J'  O  A'.  V.V  V.V 


★ 

Since  tn,  and  hence  mn,  are  asymptotic  to  r(An).  We  now  have 

J(V";"n‘V)»f)  **  *'1d+(2c)"1)2p(2p-l)‘1r(An)/n. 
n 

In  a  similar  way,  it  can  be  shown  that 

J(^n(*;mn),f)  -  *_12p(2p-l)“1r(2n)/n. 

The  rest  of  Theorem  4  now  follows  upon  using  the  facts  m*  ~  r(An), 

n 

mn  ~  r(2n),  and  r  is  regularly  varying  of  index  l/(2p). 

To  determine  the  amount  of  improvement  (if  any)  that  is  possible  with 
an  ARMA  estimator,  it  would  be  desirable  to  choose  c  to  minimize  the 
limiting  ratio  of  MISEs  in  Theorem  4.  This  minimization  problem  appears  to 
be  intractable  analytically,  although  one  could  certainly  determine  the 
best  c  numerically  for  any  given  p.  In  the  following  theorem  it  is  shown 
that  the  choice  c  *  p  is  such  that  the  limiting  ratio  in  Theorem  4  is  less 
than  1  for  each  p  >  1/2.  Hence,  under  the  conditions  of  Theorem  4,  it  is 
always  possible  to  obtain  asymptotically  smaller  MISE  with  an  ARMA 

A 

estimator  than  with  fn(»;m). 

Theorem  5  Let  the  conditions  of  Theorem  4  hold  with  c  =  p.  Then,  for  each 
P  >  1/2, 

11m  J(^n( •;%,«_*), f)/J(f_(»;m  ),f)  = 
n-HB  n  n  m_  n  n 

M  n 

Bp  =  C(2p-l)Ipjp]1/(2p)(l+(2p)"1)1'1/(2p)  <  1. 

Proof:  Using  the  fact  that  (l+y)"p  e”p^  >  e“2p^  for  y>0,  it  is  easy  to 

show  that  ( 2 p-1 ) I  <  l/(2p).  Hence,  B  <  (2p)_1^2p^ (l+(2p)-1  )1-1^2p\ 

P*  P  P 
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which  is  less  than  or  equal  to  1  iff  2plog(2p)  -  (2p-l)log(2p+l)  >  0. 
Defining 

g(x)  =  xlogx  -  (x-1)  log(x+l)  for  x  >  1, 

it  is  now  sufficient  (since  g(l)  =  0)  to  show  that  g'(x)  >  0  for  x  >  1.  We 

-1, 


have  g'(x)  =  2/(x+l)-log(l+x  j  with  g  (l)  =  1  -  log2  >  0.  Since  g'^x)  0 

as  x  ♦  the  result  will  follow  if  g'  is  monotone  decreasing  on  (1,*).  -We 
have  g"(x)  *  -2(x+l)‘2  +x_1  -  (x+1)”1,  which  is  less  than  0  on  (1,®),  and 
the  proof  is  complete. 

Recalling  the  ej-transform  motivation  for  ARMA  estimators,  it  is  of 

interest  to  determine  if  o  *  a  .  -  /*  satisfies  m(l-a)-»p  when  (5.1)  holds. 

m  m+i  m  m 


As  shown  by  Seneta,  pp.  2-7,  R_p  is  regularly  varying  with  index  -p  if  and 


only  if  it  satisfies,  for  all  t  sufficiently  large. 


R_p(t)  *  t"pexp(n(t)  +  /]je(x)x  -1dx). 


(5.4) 


where  n  is  a  bounded,  measurable  function  on  [B,®)  such  that  n(t)-*cQ  as 


t-*®  (|c0|<®),  and  c  is  a  continuous  function  on  [B,®)  such  that  e(t)<>  as 
t-*®.  Now,  if  the  function  n  in  (5.4)  is  replaced  by  a  constant,  it  is  easy 


to  show  that  m(l-R_p(m+l)/R_p(m) )+p  as  m-»®.  Therefore,  Theorem  5  implies 


that  under  a  slightly  stronger  condition  on  the  's  than  imposed  by  (5.1), 

J 


the  MISE  of  an  ARMA  estimator  with  asymptotically  smaller 


than  that  of  f  ( • ;m) .  Since  (5.4)  does  not  decay  geometrically,  the  last 


result  gives  a  good  indication  of  the  versatility  of  the  e^transform  based 
ARMA  estimator. 

We  close  this  section  with  a  corollary  that  extends  the  results  of 
Sections  4  and  5. 


t  l'i.4  l*V 


J  1 .1 '  i J  ( 


Corollary  1  Suppose  the  *.’s  in  Theorems  1-5  are  replaced  by  (-1)J4>.  for 

J  J 

j  =  1,2,...,  and  the  respective  values  for  a  are  replaced  by  -a.  Then  the 
results  of  the  five  theorems  are  unchanged. 

i 

Proof:  Considering  expression  (3.1),  it  is  clear  that  0(fn( • ;m,a) ,f)  = 

A 

J(fp( *;m,-a) ,f*),  where  f  and  f*  have  Fourier  coefficients,  respectively, 
♦j  and  (-l)J  4>j,  j  ■  1,2,...  . 


6.  Densities  That  Are  Smooth  at  One  Endpoint  but  Not  the  Other 


In  this  section  it  is  shown  that  the  results  of  Section  5  are 
applicable  under  simple  qualitative  conditions  concerning  the  smoothness  of 
f.  When  these  conditions  hold,  it  is  found  that  a  result  like  that  in 

A 

Theorem  5  remains  valid  even  when  fn(»;m)  is  replaced  by  the  very  best 
estimator  of  the  form  (4.4).  To  obtain  this  somewhat  surprising  result,  we 
first  state  the  following  lemma.  The  proof  is  analogous  to  that  of  the 
first  theorem  in  Section  3  of  Watson  and  Leadbetter  (1963),  and  is  thus 
omitted. 

Lemma  4  Suppose  that  for  some  p  >  1/2  jp|$. | -HC1^2>0  as  j*®.  Then 

Jn,opt  ~  (l+t2p)-1dt, 

.where  Jp  ^  is  defined  by  (4.5). 

The  following  theorem  will  allow  us  to  obtain  the  most  interesting 
results  of  this  paper. 

Theorem  6  Suppose  that  f  has  2k  derivatives  (k>l)  on  [0,n]  with  f^2k^ 
square  integrable.  (The  quantities  f^r^(0)  and  f(r^(n)  are  defined  to  be 
f^(0+)  and  f^U-),  respectively.)  If  f^r^(0)  =  f^(*)  =  0  for 
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r  =  1,  3,...,2k-3,  f(2k_1)(ii)  =  0  and  f(2k_1)(0)  *  0,  then  (with 
m(l-am)^2k) 


0<f>;«vv 

n 


r(2k-l)/n« , (l/2k). 


(0)|' 


(6.1) 


0(fn(-;m*,am*),f)/J(fn(.;mn),f)  = 
n 

[(4k“1)I2k,2k]1/(4k)(1+(4k)"1)1"1/(4k)  <  l*  and  (6.2) 

M#  J(V#;mn*°m;  >’f)/Jn,opt  = 

Ck[21/(4k)n/(4ksin(it/(4k)))T1  <  1  ,  (6.3) 

where 

Ck  =  [(4k+l)/(4k-l)]1'1/(4k)(8kl2kj2k)1/(4k)  . 

If  f<r)(0)  =  f(r)(n)  »  0  for  r  -  1,  3 . 2k-3,  f(2k_1^(0)  =  0,  f(2k-1)(n) 

*  0,  and  m(l+am)+2k,  then  the  result  of  the  theorem  is  the  same  except  that 

f (2lc-l ) (o)  is  repiace(j  by  f(2k”1)(n). 


Proof:  Using  integration  by  parts  we  have 

(-l)k4j  *  j-2k[f(2k'1)(0)  +  tj>2k]  ,  where 

♦j>2k  =  r0  f^2khu)cosjudu. 

Since  f^2k^  is  square  integrate,  2k-*0  as  j-*».  It  follows  that  the 

/>Lr  /  Al-  1  \ 

$>.'s  are  regularly  varying  with  index  -2k  and  that  jtKU 

J  I 


.  ^  fi«-ii(0)  as 

J 


Results  (6.1)  and  (6.2)  now  follow  upon  applying  Theorems  4  and  5, 
respectively.  The  limit  in  (6.3)  is  a  consequence  of  Theorem  4,  Lemma  4, 
and  the  fact  that 

/q( l+t^k )_1dt  *  it[4ksin(«/(4k))]-1  . 

It  is  argued  in  the  Appendix  that  the  limit  in  (6.3)  is  less  than  1.  The 
case  where  f^lc"1^(0)  =  0  and  f^k-l)(1t)  *  q  f0y|0WS  from  Corollary  1. 

The  special  case  k  =  1  in  Theorem  6  is  worth  discussing.  In  this 
case,  we  have  either  f * (0)  *  0,  f'(it)  =  0  or  f ' (0)  =  0,  f'(u)  *  0.  In 
either  situation,  the  density  tends  smoothly  to  its  limit  at  one  endpoint 
but  not  at  the  other;  exponential -like  densities  are  one  example  of  such 

A 

behavior.  The  poor  performance  of  the  Fourier  series  estimator  fn(0;m) 
when  f ' (0)  *  0  has  been  noted  by  Buckland  (1985)  in  the  setting  of  line 
transect  sampling.  Theorem  6  shows  that  In  such  cases  the  MISE  of  an  ARMA 
estimator  is  (for  large  enough  n)  smaller  than  that  of  the  very  best 
Fourier  series  estimator  of  the  form  (4.4)  (which,  of  course,  includes 

A 

f n( - ;m)  as  a  special  case). 

Table  1  shows  how  much  of  an  improvement  in  MISE  is  obtained  with  the 
ARMA  estimator  in  certain  cases  where  the  's  decay  algebraically.  The 

J 

cases  p  =  2,  4,  10,  20  correspond  to  k  =  1,  2,  5,  10  in  Theorem  6.  Note 
that  when  p  =  2  (the  situation  discussed  in  the  previous  paragraph)  the 
asymptotic  MISE  of  the  ARMA  estimator  is  only  about  64%  ana  77%  of  that 

A 

for,  respectively,  fn(*;m)  and  the  optimum  Fourier  series  estimator.  It 
is  also  Interesting  that  the  ARMA  estimator  uses,  in  the  limit,  only  51%  as 

A 

many  Fourier  coefficients  as  does  f  (»;m). 


7.  Choice  of  Smoothing  Parameter  by  Cross-Validation 


In  the  practice  of  density  estimation,  one  must  usually  select 
smoothing  parameters  via  some  data-driven  method.  As  a  result,  data-based 
density  estimators  are  not  as  efficient  (at  least  in  small  samples)  as 
theory  would  suggest.  Hence,  it  is  not  realistic  to  expect  that  the  MISE 
improvement  discussed  in  previous  sections  is  fully  attainable  in  practice, 
except  perhaps  in  very  large  samples.  However,  some  improvement  in  small 
samples  seems  likely  if  the  ARMA  estimator's  smoothing  parameter  is 
reasonably  chosen.  This  is  borne  out  in  the  simulation  study  of  this 
section. 

One  means  of  choosing  the  smoothing  parameter,  (m,a),  of  the  ARMA 
estimator  is  the  cross-val idatory  method  introduced  by  Rudemo  (1982)  and 
Bowman  (1984).  In  our  setting,  this  method  chooses  the  pair  (m,a)  that 
minimizes 


R(m,o)  =  JJ  f*(x;m,a)dx  -  (2/n)  J  f n ,i ^Xi »m*a) 


(7.1) 


where  f  .  indicates  the  estimator  calculated  by  deleting  the  data  value 
n,i 

A 

.  Rudemo  (1982)  showed  that  R(m,a)  is  an  unbiased  estimator  of  the  risk 


R(m,«)  =  J(fn(-;m,a),f)  -  JJ f2(x)dx. 

A  number  of  results  now  exist  showing  that  density  estimates  chosen  by 
cross-validation  are  asymptotically  efficient;  see,  for  example.  Hall 
(1983,  1985),  Stone  (1984),  and  Hall  and  Marron  (1985). 


To  Investigate  the  behavior  of  cross-vall dated  ARMA  estimates,  a  small 
simulation  study  Mas  conducted.  The  density  considered  Mas 

f(x)  -  2e_2x(l+e'4(lt'x))(l-«r4V1,  0  <  x  <  *,  (7.2) 

Mhich  has  algebraic  Fourier  coefficients 

♦j  -  (l+U^)2)'1.  j  *  1.2 . 

This  density  Is  of  Interest  since  It  has  f  (0)  -  -2(l-e'4*)  and  f  (*)  ■  0, 
and  thus  satisfies  the  conditions  of  Theorem  6. 

For  (7.2)  and  n  *  50,  the  minimum  MISE  among  ARMA  estimators  is 
.00633.  This  minimum  occurs  at  (m,«)  ■  (1,  .64).  Among  Fourier  series 

A 

estimators  fn(*;m),  the  optimum  m  and  MISE  are  5  and  .04160.  Since 
.00633/. 0416  *  .152,  Me  see  that  the  asymptotic  relative  efficiency  of  .64 
from  Table  1  understates  the  Improved  efficiency  of  the  optimum  ARMA 
estimator  at  n  *  50. 

TMenty  Independent  random  samples  of  size  n  *  50  Mere  generated  from 
the  density  (7.2).  This  Mas  done  by  generating  values  from  the  exponential 
density  g(x)  *  2e"2x  ^(x),  and  using  the  fact  that,  if  Y  has  density 

g,  then 

*  *  n(0,*)<V> 

has  density  (7.2).  Since  little  of  the  mass  of  g  is  larger  than  x,  the 
graphs  of  f  and  g  on  (0,x)  are  virtually  indistinguishable. 


29 


For  each  of  the  twenty  data  sets,  the  minlmizer  of  R  (defined  by 
(7.1))  for  0  <  a  <  1,  1  <  m  <  20  was  approximated.  Also,  the  minimlzer  of 

A 

R(m,0)  for  1  <  m  <  20  was  determined.  This  latter  value  of  m  is  simply  a 

A 

cross-val idatory  choice  of  the  smoothing  parameter  of  f  (*;m)  (see  Hart 
1985  and  Diggle  and  Hall  1986  for  more  on  this  subject).  The  Integrated 
squared  errors  of  the  cross-validated  ARMA  and  Fourier  series  estimates 
were  determined  for  each  data  set.  Denote  these  two  ISEs  IA  and  Ip, 
respectively. 

The  results  of  the  simulation  are  summarized  In  Table  2.  The  fact 
that  average  1^  was  a  bit  larger  than  average  Ip  Is  misleading.  In  16  of 
the  20  cases,  the  ratio  I^/Ip  was  between  .151  and  .690.  Note  the  trimmed 
mean  and  the  confidence  Interval  for  the  median  of  IA/Ip  in  Table  2.  These 
more  accurately  reflect  the  overall  performance  of  the  two  cross-validated 
density  estimates.  From  the  16  cases  in  which  1A  was  less  than  Ip,  a 
typical  comparison  of  the  two  estimates  is  given  in  Figure  1.  The 
qualitative  improvements  obtained  with  the  ARMA  estimate  are  a  better 
estimate  of  f(0)  and  an  absence  of  spurious  bumps.  See  Hart  and  Gray 
(1985)  for  further  discussion  of  the  qualitative  properties  of  ARMA 
approximators . 

It  is  also  important  to  point  out  what  happened  in  the  four  cases 
where  IA  >  Ip.  Figure  2  is  a  plot  of  the  20  values  of  (m,a)  chosen  by 
cross-validation.  In  the  cases  where  IA  >  Ip,  cross-validation  chose 
too  large  an  a  and/or  too  large  an  m.  When  a  Is  too  near  1,  the  ARMA 
estimate  tends  to  be  too  large  near  0,  thus  inflating  the  estimate's 
Integrated  squared  error.  That  cross-val idation  would 
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occasionally  choose  too  rough  an  ARMA  estimate  Is  not  unexpected.  It  Is 
well  known  from  other  settings  that  In  small  samples  cross-validation  tends 
to  drastically  undersmooth  around  5-20%  of  the  time  (see,  e.g..  Hart  1985). 

Clearly,  further  experimentation  Is  needed  to  determine  how 
efficiently  cross-validation  smooths  ARMA  estimates.  Some  very  recent  work 
of  Scott  and  Terrell  (1986)  shows  that  a  biased  version  of  cross-validation 
provides  more  efficient  kernel  estimators  In  moderate  and  large  samples. 
Such  an  Idea  1$  also  worth  pursuing  In  the  setting  of  ARMA  estimators.  For 
example,  a  modification  of  R(m,a)  that  places  a  more  severe  penalty  on 
large  values  of  ja|  would  discourage  the  occasional  undersmoothing  observed 
In  the  simulation  study. 


8.  Concluding  Remarks 

Me  have  shown  that  under  a  variety  of  conditions,  a  density  estimator 
having  the  rational  form  of  an  ARMA  spectrum  can  yield  an  Improvement  In 
MISE  over  the  very  best  Fourier  series  estimator  of  the  form  (4.4). 
Furthermore,  If  Its  smoothing  parameter  Is  chosen  properly,  the  ARMA 
estimator  never  has  larger  MISE  than  that  of  the  simple  Fourier  series 

A 

estimator  f  (»;m).  Although  further  work  Is  needed  on  the  data-based 
smoothing  of  ARMA  estimates,  the  results  of  Section  7  are  fairly 
encouraging  with  regard  to  cross-validated  smoothing. 

The  Ideas  of  this  paper  could  also  be  applied  to  other  density 
estimators  of  the  type 

f  (x)  ■  (1/x)  [1+2  y  w.  cosjx]  . 
m  j«l  J  J 


The  form  (2.4)  1  allied  lately  suggests  “jackknifed*  versions  of  f  .  So  long 
as  the  truncation  bias  of  f  Is  not  dominated  by  the  bias  due  to  the 
Wj's,  results  analogous  to.  for  example,  Theorem  5  could  be  established 
for  a  jackknifed  f  . 

Finally,  our  results  could  also  be  extended  s owe  what  by  considering 
estimators  based  on  a  sine-cosine  basis  and  allowing  a  to  be 
complex-valued.  In  fact,  the  results  of  Hart  and  Gray  (1985)  apply  to  the 
bias  of  such  estimators. 
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Appendix 


Proof  of  (6.3):  As  stated  In  the  proof  of  Theorem  6,  we  need  only 
show  that  the  limit  In  (6.3)  Is  less  than  1.  For  k  *  1  or  2,  the 

inequality  follows  from  Table  1.  For  k  ■  3,4 .  it  Is  sufficient  to  show 

that  Ck2_1/*4k^  <  1,  since  0  <  x-1sinx  <  1  for  0  <  x  <  */12.  Now, 

c^2-l/(4k)  ,  4k(4k-l)‘1(l+(4k)'1)1-1/(4k)[(4k-l)I2k 

<4k(4k-l)‘1(l+(*k)_1)1'1/^4k)(*k)‘1/(4k)  ,  as 

In  the  proof  of  Theorem  5.  The  last  quantity  is  less  than  or  equal  to  1 
for  k  *  3,  4,...  if 

g(x)  *  xlog(x-l)  -  (x-1)  log(x+l)  >  0 

for  x  >  12.  The  rest  of  the  proof  proceeds  exactly  as  In  the  proof  of 
Theorem  5.  (Note  that  g ( 1 2)  >  0.) 

Calculation  of  numbers  in  Table  1:  The  requisite  quantities  can  all 
be  calculated  analytically  except  for 

(2p-l)I  -  2  -  ( 2  p )  ” 1  -  2(2p-l  )epF  (  p)  , 

P.P  P 

where  Ek(y)  *  /*  e~ytt"kdt  is  the  so  called  exponential  Integral.  The 
table  values  were  obtained  by  using  the  approximation  of  either  Ep(p)  or 
epEp(p)  given  in  Abramowltz  and  Stegun  (1972). 
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Table  1.  Asymptotic  MISE  of  ARMA  and  Fourier  Series  Estimators  for 
Densities  With  Algebraically  Decaying  Fourier  Coefficients 


Notes: 


(It  is  assumed  that  as  j-*».) 


Type  of  Estimator 

p 

ARMA 

Optimum 

Fourier  Series 

Fourier  Series 
with  0-1 
Weights 

1 

1.3579 

1.5708 

2 

2 

.8534 

1.1107 

4/3 

3 

.7968 

1.0472 

6/5 

4 

.7890 

1.0262 

8/7 

.8335 

1.0041 

20/19 

BE 

.8826 

1.0010 

40/39 

For  a  given  p  and  estimator,  a  table  value  is  the  limit  of 
nl-l/(2p)Bp  MI$E,  .  w(2k)-1/(2p^.  Details  of  how 

the  values  were  obtained  are  given  in  the  Appendix.  For  a  given  p 
the  limiting  ratio  of  ARMA  truncation  point  (mn)  to  Fourier 
series  truncation  point  (mn)  is  (l+^p)"1)"1  Ap/FSp,  where  ^p  and 
FSp  are  the  table  values  in,  respectively,  the  first  and  third 
columns  above. 


Table  2.  Summary  of  Simulation  Study 


Type  of  Estimate 


ARMA 

Fourier  Series 

Average 

ISE 

.0542 

.0522 

Median 

ISE 

.0183 

.0354 

Trimmed 

mean  ISE 

.0287 

.0390 

The  trimmed  means  exclude  the  three  (out  of  20)  largest  values  of 
ISE.  A  95%  confidence  interval  for  the  median  of  I^/Ip  is 
(.3607,  .6581),  where  and  Ip  are,  respectively,  the  ISEs  of 
cross-vali dated  ARMA  and  Fourier  series  estimates. 


Captions  for  Figures 

Figure  1.  Wrapped  Exponential  Density,  ARMA  Estimate,  and  Fourier 
Series  Estimate.  The  solid  curve  is  the  density  (7.2).  The  ARMA  estimate 
has  the  higher  value  at  0  and  no  spurious  bumps.  The  two  estimates  were 
calculated  from  the  same  set  of  data,  each  being  fitted  by  cross- 
validation. 

Figure  2.  Distribution  of  Smoothing  Parameters  in  Simulation  Study. 
The  smoothing  parameters  were  chosen  by  cross-validation.  The  four  largest 
values  of  m  correspond  to  the  only  cases  where  the  ISE  of  the  ARMA  estimate 
was  larger  than  that  of  the  Fourier  series  estimate.  The  MISE  optimum 
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